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Institutional Researchers as Leaders in Policy: Perspective and Possibilities 



ABSTRACT 



This paper presents the results of a research study that investigated institutional researchers' 
perceptions regarding their effectiveness in influencing policy changes at their institution. 

Data were obtained from a mailed survey sent to 304 researchers in the Northeast; a response 
rate of 73 percent was achieved. Results revealed that those whose work is used in executive 
decision-making; who include policy recommendations in reports; conduct follow-up studies on 
the impact of their work; have a doctorate, are part of a strong, professional network; and 
describe their positions as challenging perceive themselves as more effective. The discriminant 
function including these variables explained 48 percent of the variance and accurately predicted 
the perceived policy effectiveness for 84 percent of the participants. 
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Institutional Researchers as Leaders in Policy: Perspective and Possibilities 

Introduction 

Purpose . This paper presents the results of a research study that investigated institutional 
researchers' perceptions regarding their leadership role in higher education planning and policy. 
The paper focuses primarily on institutional researchers' perceived effectiveness in influencing 
policy changes at their institution. The major research questions are: 

• How effective are institutional researchers in influencing policy? and 

• What factors make institutional researchers more effective in the policy arena? 

Answers to these questions offer insights that can be used to enhance the effectiveness of all 
practicing institutional researchers. The study also documents institutional researchers' 
perspectives regarding job-related rewards; challenges encountered in seeking to establish 
influential roles in decision-making; and strategies utilized for coping with these challenges. 

In the context of this study, effectiveness is based on participants' responses to the question, 
"Has your work effected program/policy changes at your institution?" Those who reported that 
their work 'often' or 'very frequently' effected program policy changes at their institution are 
classified as more effective, while those who reported that their work 'almost never', 'not often' or 
'sometimes' effected program policy changes at their institution are classified as less effective. 

Limitation of the Study . This research is a study of perceived effectiveness. Effectiveness 
is defined by respondents' views of their own effectiveness. These subjective perceptions may 
differ from other measures, such as supervisors' assessments. Future research might investigate 
the relationship between institutional researchers' judgments and their supervisors' evaluations 
with respect to policy effectiveness. 
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Review of the Literature . Researchers in the field of higher education recognize the 
significance of policy effectiveness for the institutional research profession. In their study of 
perceptions of effectiveness, Knight, Moore and Coperthwaite (1997) found that institutional 
researchers were more likely to consider themselves to be effective if they possessed the 
following characteristics: were employed in the field for a greater number of years; held the 
doctorate; were associate directors; and reported directly to the institution's president. 

Sanford (1995) outlined four main challenges that institutional researchers need to address 
to ensure success in tomorrow's academic environment: adaptability to change, access to data 
with modem technology, better comparative data, and the policy analysis / technology resource 
dilemma. Keller (1995) contends that institutional researchers will need to work closely with 
administrators and provide information about trends, social changes and regional conditions. He 
claims that superior academic management will be more closely tied than ever to quality 
institutional research. 

During the last decade, several researchers have recommended changes to ensure that 
institutional researchers will be effective in policy development. For example, Matier, Sidle, and 
Hurst (1994) advocate expanding the scope of institutional research to encompass the roles of 
information architect, change agent and consultant of choice within higher education institutions. 
Delaney (1997) recommends enhancing the capacity for conducting complex research studies; 
creating and supporting high-level audiences for institutional research studies; and expanding the 
focus of institutional research studies to include relevant factors and external trends. Lohmann 
(1998) proposes that institutional researchers become competent in policy debate; develop a 
power base; increase the level of environmental scanning; participate in the strategic planning 
process; and shift studies from mere reporting to timely research on pressing issues. 
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Methodology 

Data Source . Data for this study are based on results from a mailed survey sent to 304 
institutional researchers in the Northeast; 221 returned completed surveys yielding a response 
rate of 73 percent. The respondent group reflects the demographic, educational and professional 
diversity of the institutional research profession. Of the 221 respondents, 41 percent are male 
and 59 percent are female; 40 percent possess a doctorate; 42 percent have a master’s degree; 
and 18 percent hold a bachelor’s degree. Respondents represent a range of professional 
positions. Eleven percent hold titles at the level of dean to vice-president; 50 percent are 
directors; 10 percent are associates; 16 percent are analysts, coordinators or mangers; and 13 
percent are assistants or research and technical specialists. 

Participants represent a range of experience in institutional research from less then one year 
to 28 years; the mean and median number of years are 9 and 8 respectively. With regard to type 
of institution, 33 percent have spent most of their career at a university, 28 percent at a four-year 
college, and 15 percent at a two-year college. The largest proportion, 29 percent have worked 
primarily at a public institution compared with 25 percent at a private, non-religious institution 
and 15 percent at a private religious institution. 

Analytical Techniques . Data analyses involved both qualitative and quantitative techniques. 
The quantitative techniques included Chi-square, t-tests, correlation and discriminant function 
analyses. Chi-square analyses examined variation in perceived policy effectiveness by gender, 
current professional position, level of education and type of institution. Correlation and t-tests 
were utilized to investigate relationships between perceived effectiveness in policy and variation 
in job rewards, job challenges, leadership experience, desire for enhanced skills and mentor 
experience. Discriminant analysis identified predictors of perceived effectiveness in policy. 
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Results 

Role in Policy . In describing their roles within the institutions, 47 percent of the 
institutional researchers in this study reported they 'often' or 'very frequently' serve on planning 
and policy committees. Approximately one-third or more 'often' or 'very frequently' present their 
work at executive level meetings; collaborate with others in program development and initiate 
discussions on program planning and policy. However, only 28 percent are 'often' or 'very 
frequently' consulted on impending policy changes, and only 12 percent 'often' or 'very 
frequently' conduct follow-up studies on the impact of their work. 

With respect to the use and influence of their work, 86 and 72 percent respectively reported 
that their work 'often' or 'very frequently' is disseminated at the vice presidential and presidential 
levels and is used in executive decision-making. However, only 31 percent reported that their 
work 'often' or 'very frequently' includes policy recommendations. Finally, only 49 percent 
reported that their work 'often' or 'very frequently' has effected program/policy changes. 

Correlation analyses were conducted to investigate the relationship between characteristics 
of institutional researchers, their practice and effectiveness in influencing policy. The two 
strongest correlates are: work is used in executive decision-making (r =.72, p < .001) and 
research reports include policy recommendations (r =.51, p < .001). Next in order, institutional 
researchers conduct follow-up studies on the impact of their work (r =.48, p < .001) and work is 
disseminated at the vice-presidential or presidential level (r =.48, p < .001). Also significant is 
the fact that more effective institutional researchers experience challenging leadership 
opportunities in their current position (r =.37, p < .001). 

T-test analyses were also conducted to identify significant differences between more and 
less effective institutional researchers both in the roles they assume and in the use of their work 
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at their institution. These differences are displayed in Table 1. In terms of roles, more effective 
institutional researches are more frequently consulted on impending policy changes; present their 
work at executive meetings; and conduct follow-up studies on the impact of their work. In 
addition, their work more frequently includes policy recommendations and is more frequently 
disseminated and used at executive levels within the institution. 



Table 1 

Significant Differences in Roles Assumed by More and Less 
Effective Institutional Researchers 



Means 



Role 


More 


Less 


Difference 


t Ratio 


Are consulted on policy changes 


3.36 


2.44 


.92 


5 94 *** 


Present work at executive meetings 


3.53 


2.66 


.87 


5 12*** 


Conduct follow-up studies 


2.85 


2.00 


.85 


6.18*** 


Collaborate in program development 


3.37 


2.60 


.77 


5 2 o*** 


Serve on planning and policy committees 


3.87 


3.10 


.77 


448*** 


Initiate discussion on planning and policy 


3.10 


2.38 


.72 


4 21 *** 


Use of Work 


Work used in executive decisions 


4.64 


3.51 


1.13 


12.08*** 


Work includes policy recommendations 


3.44 


2.41 


1.03 


6.85*** 


Work disseminated at executive level 


4.80 


4.14 


.66 


6.18*** 


Note. Response scale: 1 "almost never" to 5 


"very frequently" 


*** p <_.001 



Chi-square analyses examined the relationships between institutional researchers' personal 
and professional characteristics and their effectiveness in policy. Variables examined include 
gender, level of education, current position, and type of institution in which they have spent most 
of their career. Results revealed only one significant relationship - between level of education 
and effectiveness in policy (X 2 = 6.40, p < .05). Researchers with more education were more 
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effective in influencing program/policy changes. Some 58 percent of those with a doctorate, 
compared with 47 percent of the master's degree and 34 percent of the bachelor's degree 
respondents reported their work 'often' or 'very frequently' has effected program/policy changes. 

Rewarding Aspects of Work . In addition to focusing on effectiveness in policy, the study 
also explored institutional researchers' perspectives on the rewards and challenges they 
experience in their work. Results from this phase of the study identify professional 
independence in conducting research as one of the most rewarding aspects of an institutional 
researcher's job. Ninety-four and 88 percent respectively report they 'often' or 'very frequently' 
are able to work on their own and have the freedom to decide how to do their work. Between 60 
and 81 percent 'often' or 'very frequently' are able to make decisions on their own; have the 
flexibility to set their own work priorities, and have the authority needed to get the job done. 

Results indicate that work rewards associated with independence differ significantly 
between more and less effective institutional researchers. As shown in Table 2, more effective 
institutional researchers have more independent authority to hire persons of their own choice, 



Table 2 

Job Rewards of More and Less Effective Institutional Researchers 

Means 



Job Rewards 


More 


Less 


Difference 


t Ratio 


Independent authority to hire 


3.20 


2.36 


.84 


3 go*** 


Authority to spend budget 


3.71 


3.21 


.50 


2.40* 


Authority to get the job done 


4.20 


3.79 


.41 


3 47 *** 


Authority to set research agenda 


3.75 


3.36 


.39 


2.56** 


Can reject superior's suggestions 


3.18 


2.80 


.38 


2.87** 


Budget for professional development 


3.91 


3.54 


.37 


2.38* 


Being able to work on one's own 


4.67 


4.45 


.22 


2.63** 



Note. Response scale: 1 "almost never" to 5 "very frequently" * p <..05; * * p <_.01; *** p <..001 
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to spend their budget, to set their research agenda, and to do what is necessary to get the job 
done. They also have more freedom to work on their own, to accept or reject superior's 
suggestions, and they receive more financial support for professional development. 

Leadership Experience . Since leadership qualities influence the extent to which individuals 
can effect change, this study also explored institutional researchers' potential, preparation and 
perceived support for their leadership role. Participants were asked to rate themselves, their 
education, professional associations, and the support they received with respect to developing 
their leadership role. Some 30 and 21 percent respectively rated themselves 'excellent' in terms 
of their leadership potential and effectiveness as a leader when given the opportunity. With 
regard to support for their leadership role, 'excellent' ratings ranged from 21 to 30 percent 
respectively for the support received from male superiors and from female subordinates. Only 
19 percent reported 'excellent' ratings for leadership opportunities in their current position. Even 
fewer, 13 percent, gave 'excellent' ratings for leadership development through professional 
associations. 

As illustrated in Table 3, more and less effective institutional researchers differ significantly 
on four leadership variables. In terms of self-ratings, more effective institutional researchers 
report higher ratings for their own potential as leaders and for their effectiveness as leaders when 
given the opportunity. Further, more effective institutional researchers rate their current position 
higher in terms of leadership opportunities and they offer higher ratings for the support received 
for their leadership role from male subordinates. These findings document the important 
relationship between leadership and perceived policy effectiveness and suggest that efforts to 
develop institutional researchers' leadership potential may result in greater policy effectiveness. 
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Table 3 

Differences in Leadership Potential and Experience of 
More and Less Effective Institutional Researchers 



Means 



Leadership Variable 


More 


Less 


Difference 


t Ratio 


Leadership opportunities 


3.76 


3.36 


.40 


2.72** 


Your potential as a leader 


4.23 


3.85 


.38 


3.56*** 


Support from male subordinates 


4.03 


3.70 


.33 


2.12* 


Effectiveness as a leader 


4.23 


3.96 


.27 


3.43*** 


Note. Response scale: 1 "very poor" to 5 


"excellent" 


* p <_.05; 


* * p < .01; *** 


p^.001 



Participants in this study also offered their opinions regarding the most important qualities a 
person needs to be an effective leader. As reflected in the following comments, they referred to 
qualities of character and personality, the capacity for vision, and competence in relevant skills. 
In their words, effective leaders possess the following traits: 

Qualities of Character 

• trustworthy, honest, able to communicate and see a vision/grand plan; 

• have integrity, compassion, a strong intellect and a willingness to listen; 

• have the courage to stand by convictions, modeling good management; and 

• possess integrity, character, ... the ability to stand alone if necessary, and 
the ability to communicate intellectually and emotionally with others; 

Personality Traits 

• knowledge, empathy, confidence that can be projected outward; 

• intelligence, the ability to listen, care about people, and the capacity for hard work; 

• wisdom, experience, interest in others; 
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• the ability to understand both sides of an issue and show fairness; and 

• a sense of humor, vision, intelligence, and the capacity to see the big picture; 

Capacity for Vision 

• the ability to see a distance ahead, set goals, and establish steps to reach these goals; 

• a vision of the organization they want to create and some idea of how to achieve it; 

• a combination of vision and strong management sense and the ability to motivate others; 

• a “vision” for goals to be achieved, . . . and the ability to connect with others to bring 
their thoughts, creativity, and energy to this process; 

Skills 

• ability to listen and to understand what others expect and read between the lines; 

• mastery of the skills needed for the position, ability to plan, flexibility, and 
self-confidence; 

• must be in tune with others, able to listen and share information, able to organize, 
delegate, and effectively hold staff meetings; 

• excellent listening skills, drawing on each person's strengths to add to the teams' efforts; 

• the ability to hire staff with complementary skills, encourage growth and 
development, protect when necessary, support when protection is not possible; 

• a dual emphasis on the task and the people doing it - initiating structure and 
showing consideration - collaborative teamwork; and 

• substantive competence in his or her profession, interpersonal skills, 

high ethical standards, a participatory management style, trustworthiness, and creativity. 
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Challenging Professional Opportunities . When asked to what extent their current position 
provides challenging opportunities in various areas, 60 percent of the study participants reported 
'very much' for computer technology, followed by 41 percent for intellectual reasoning, 33 
percent for research design and statistical analysis and 27 percent for higher education policy. 

In contrast, only 20 percent or fewer reported that their current position offered 'very much' of a 
challenge in the areas of leadership, management and training opportunities in research. 

Comparative analyses identified statistically significant differences between more and less 
effective institutional researchers in the extent to which they experienced various challenging 
opportunities in their current position. These differences are documented in Table 4. As shown, 
more effective institutional researchers report more challenging opportunities in leadership, 
higher education policy, and management, as well as in intellectual reasoning, research design 
and statistical analysis. 



Table 4 

Significant Differences in Challenging Opportunities of 
More and Less Effective Institutional Research 



Means 



Challenging Opportunity 


More 


Less 


Difference 


t Ratio 


Leadership 


3.75 


3.13 


.62 


4.36*** 


Higher education policy 


3.97 


3.42 


.55 


3.76*** 


Management 


3.81 


3.28 


.53 


3.75*** 


Research design 


3.99 


3.57 


.42 


2.80** 


Statistical analysis 


4.02 


3.67 


.35 


2.42* 


Intellectual reasoning 


4.34 


4.01 


.33 


2.97** 


Training opportunities in research 


3.16 


2.83 


.33 


2.02* 


Note. Response scale: 1 "not at all" to 5 


"very much" 


* p <..05; 


* * p < 01; *** 


p ^.001 
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Interest in Skill Development . When asked to identify the skills they would like to develop, 
56 and 48 percent respectively expressed interest in developing their statistical and technological 
skills. Next in order, 38 percent expressed an interest in enhancing their leadership skills and 37 
reported an interest in developing their skills in each of the following areas: analysis, budget and 
research knowledge. 

Chi-square analyses identified a statistically significant difference between more and less 
effective institutional researchers in terms of one skill enhancement area - analytical skills. A 
higher 60 percent of the less effective, compared with only 40 percent of the more effective, 
institutional researchers expressed an interest in developing analytical skills (X2 = 4.20, p < .05). 

Resources and Strategies for Success . The study also explored what resources and strategies 
institutional researchers use to overcome obstacles and achieve professional success. The vast 
majority, 78 percent, reported they have had a mentor. Sixty-six percent have sought other 
professionals in similar positions to advise them, and 59 percent have a strong professional 
network. The majority reported that their mentors 'frequently' or 'almost always' engaged in the 
following activities: communicating - listening and responding; motivating and encouraging 
growth; advising on substantive work issues; role modeling, and validating. No statistically 
significant differences were found between policy effectiveness and the role of mentors. 

Statistically significant differences were found between policy effectiveness and researchers' 
initiative in seeking out other professionals to advise them (X 2 = 8.23, p < .05) and involvement 
in a strong network of other professionals (X 2 = 8.47, p < .05). Sixty-eight percent of the more 
effective, compared with only 53 percent of the less effective institutional researchers, report 
they sought both male and female professionals to advise them. Similarly, 67 percent of the 
more effective, compared with only 49 percent of the less effective institutional researchers, 
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reported that they were part of a strong network of female and male professionals. These results 
indicate that more effective institutional researchers reach out to and rely on a network of 
support from both male and female colleagues. 

Predictors of Policy Effectiveness . Discriminant analysis was employed to determine which 
combination of variables would predict institutional researchers' effectiveness in influencing 
policy. Table 5 displays the results from this analysis. The discriminant function coefficients 
indicate the relative weights for each variable found to be a significant predictor of effectiveness. 

Table 5 

Discriminant Analysis Results: 

Predicting Policy Effectiveness in Institutional Research 



Predictors 


Standardized 
Discriminant 
Function Coefficients 


Percent 

Correctly 

Classified 


Work used in executive decision-making 


.85 


84% 


Conduct follow-up studies on impact of work 


.22 




Seek other professionals to advise 


.14 




Hold a challenging professional position 


.14 




Attain a high level of education 


.10 




Work includes policy recommendations 


.10 




Are part of a strong professional network 


.02 




Canonical Correlation 

- 77 


.69 





X 2 = 127.98; 7 df; p < .001 



As shown in Table 5, the strongest predictor of policy effectiveness is that work is used in 
executive decision-making. Next in order are specific behaviors of institutional researchers 
including conducting follow-up studies on the impact of the work; seeking out other professionals 
for advice; and holding a challenging professional position. In this analysis, challenging 
professional position is based on a mean response to the question, "To what extent does your 
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current position provide challenging opportunities in intellectual reasoning, leadership and 
management?" Including policy recommendations in one's work, attaining a high level of 
education, and being part of a strong network of professionals are also significant predictors of 
policy effectiveness. 

These results indicate that institutional researchers who are able to introduce their work 
into executive decision-making and who take a pro-active role with respect to their own work are 
likely to be more successful in influencing program and policy changes. This pro-active role 
includes formulating policy recommendations based on research findings and conducting follow- 
up studies to determine if the research had any impact. Other academic and social factors also 
make a difference. Researchers who hold a doctorate and those who seek advice and support from 
other professionals are also more effective in influencing planning and policy. 

The discriminant function including these seven variables accurately predicted the policy 
effectiveness of 84 percent of the respondents. The canonical correlation of .69 indicates that this 
function explains 48 percent of the variance in institutional researchers' effectiveness in 
influencing program and policy changes at their institution. 

Discussion 

Results from this research identify qualities of institutional researchers, characteristics of 
their positions, and specific professional behaviors significantly related to accomplishing 
institutional program and policy changes based on research. These results provide a framework 
for developing strategies to further enhance the policy role of individual institutional researchers 
and the institutional research profession. 

This study confirms results from previous research that level of education has a significant 
impact on policy effectiveness. Bivariate and multivariate analyses revealed that institutional 
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researchers with a doctorate reported significantly more often that their work had resulted in 
program or policy changes at their institution. Previously, Knight, Moore and Coperthwaite 
(1997) also discovered that institutional researchers with a doctorate were more likely to consider 
themselves to be effective. Delaney (1997) found the presence of an institutional research 
director with a doctorate to be significantly related to involvement in planning and policy 
development. 

Findings from this study also revealed an interesting insight with regard to the relationship 
between policy effectiveness and how institutional researchers conduct their professional lives 
and their work. Institutional researchers who seek other professionals to advise them and who 
are part of a strong professional network report significantly more often that their work has 
achieved program and policy changes at their institution. 

Finally, this study identified three professional behaviors significantly related to policy 
effectiveness. The first and strongest is that work is used in executive decision-making. This 
requires collaboration between the institutional researcher and the decision-makers. The other 
two behaviors are within the discretion of the institutional researcher - including policy 
recommendations in research reports and conducting follow-up studies on the impact of research 
studies. 

The identification of these behavioral predictors of policy effectiveness supports the 
findings and recommendations of several previous researchers. Keller (1995) recommended that 
institutional researchers work more closely with administrators to realize successful management 
in tomorrow's academic environment. Matier, Sidle and Hurst (1994) advocated that institutional 
researchers become information architects, change agents and consultants of choice with higher 
education institutions. Lohmann (1998) proposed that institutional researchers become 
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competent in policy debate, participate in the strategic planning process, and shift studies from 
mere reporting to timely research on critical issues. Results from this study support the validity 
of each of these recommendations and confirm their relevance to effectiveness in program 
planning and policy. 

Conclusion 

Findings from this study identify strategies for enriching institutional researchers' 
perspectives on their positions and enhancing the possibilities for success in influencing policy. 
Results show that institutional researchers who include policy recommendations in their reports; 
conduct follow-up studies on the impact of their work and whose work is used in executive 
decision-making perceive themselves to be more effective in influencing policy. Having a 
doctorate and being part of a strong network of other professionals also enhance perceived policy 
effectiveness. Further, institutional researchers who experience success in the policy arena find 
their positions challenging. These results suggest possible objectives and strategies that aspiring 
institutional researchers and professional associations may pursue to ensure that institutional 
researchers become influential leaders in policy development. 
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